I want to talk about Kubernetes for a little bit. But before we get into Kubernetes, I want to talk about containers. And not the containers that run inside of pods on Kubernetes. I want to talk about the old boring [intermodal shipping containers] (https://en.wikipedia.org/wiki/Intermodal_container) that have become a staple of the shipping industry.

These dull, 20 feet long, 8 feet 6 inch tall titans of industry don’t look like the type of invention that would help revolutionize commerce, yet that’s exactly what it did. Back in the early 1900s, the shipping industry looked very different. The norm for shipping back then was called “Breakbulk Cargo”. (It’s still in use today, but for specific cases) Breakbulk cargo or “general cargo” was cargo that was shipped in individually counted units. That might sound innocuous enough, but when you’re dealing with a ship that has 20,000 goods on it, you can imagine how inconvenient it must be to keep track of an individual PS5, for example. But in the early 1900s, dealing with cargo in this fashion was pretty commonplace.

There are a few issues with breakbulk cargo. One of the biggest issues was the speed at which you could load and unload the cargo. Dealing with breakbulk cargo is a labor-intensive process. Then along came a guy named [Malcom Mclean] (https://www.pbs.org/wgbh/theymadeamerica/whomade/mclean_hi.html).

Malcom Mclean was an American businessman and a former truck driver who was compelled to find a better solution for the shipping industry. Mclean is credited with inventing the intermodal shipping container which changed the shipping industry forever. The interesting thing about Mclean’s story was that he wasn’t the first person to try using shipping containers. Shipping containers were in use as early as 1926, almost 9 years before Mclean even finished high school. So why did Mclean’s idea for containers take off while other implementations surfed in mediocrity?

Mclean’s biggest insight into the process wasn’t just the container. It was the need for the entire industry to change in order to align practices and standards around the container. Having a container show up at a port, only to be manually unloaded a single item at a time would limit the impact that the container would have. But if ports were all standardized to expect these 20’x8’x8’6″ containers, the loading, offloading and storage process could bring an advantage that just wouldn’t be possible if ports used the same breakbulk infrastructure as they did with containers. Everything needed to change to maximize the success of the container.

Is your organization ready for containers?

Fast forward to today and to a different type of container. Kubernetes is on fire across the tech industry as companies rush to adopt it in hopes of turbo-charging their development organization’s productivity. And why shouldn’t they? Kubernetes has been touted as *the tool* to help teams feel empowered, to move faster, and deliver without interference or bottlenecks from other teams. Well, at least this is what the marketing information for Kubernetes says. But as the killer of hype trains, I have to tell you I’ve got bad news. Kubernetes can’t help you with any of that because the technology was never the thing that was stopping you in the first place.

Before Kubernetes, there was the cloud, before the cloud there were data centers, and in the data centers, we had virtual machines. These virtual machines could have been automated and orchestrated in such a way to empower developers to do all the magical things we talk about today. VMWare’s [vCenter] (https://www.vmware.com/products/vcenter.html) product was introduced in 2003. Even then vCenter provided some basic deployment and provisioning technology that, if properly exposed, could have revolutionized the speed at which developers operated. But old word patterns continue to creep in. Now, engineers still made a request to IT, but instead of that request turning into a hardware requisition, it was a VM requisition. There was definitely an improvement in speed and turnaround time, since we weren’t talking about purchase orders and delivery dates, but it still shows how the organization itself limited the opportunity that the technology provided.

This is a very common scenario. Organizational mindsets and practices are usually what gets in our way. The fear of “separation of duties”, operations teams that felt they were the only ones who cared about the safety of production, and of course the specter of change management, forced us all to continue to live in a world of gates, approvals, and go/no-go meetings. A lot of these processes have eased up today but the ideas and frameworks of old world thinking still lurk in our minds and our processes.

When a technology like Kubernetes comes around, we have to ask ourselves, “How is our current way of doing things holding back the true potential of this tool?” If you implement Kubernetes but still intend on concentrating all of the operational responsibility on to a centralized OPS team, you’ll never achieve the full potential and capability of the tool. The marketing material on Kubernetes talks about empowering development teams, but empowerment isn’t just a tool. It’s a mindset and an attitude that can only exist if the organization breathes life into it. Empowerment is the ability to make a decision. Kubernetes can never give that to an engineer. It can only give them the capability to act on that decision.

If a developer finds a bug in their software, empowerment is giving them the discretion to decide that it’s safe to release a bug fix on a Friday at 9pm. That decision will obviously have multiple competing factors before it’s finalized. But if that same engineer needs an approval from their manager, an approval from a five person change board and a VP level sign off, then the engineer isn’t empowered to do anything and running Kubernetes won’t help you.

When you look to introduce a tool like Kubernetes, you must look and see how the entire ecosystem of your organization has to change. Responsibilities might be enhanced, diluted and generalized. Software engineering teams might be thrust into the role of both software engineer and operations engineer, since it doesn’t make a lot of sense for a team to be able to brainstorm, develop and launch a service, only to turn it over to another group to manage it in production.

The operations team that previously had intimate knowledge of the various applications and services, along with their corresponding idiosyncrasies, now might need to meld into the background as they become more concerned with the underlying platform (Kubernetes) and become relatively ignorant to the applications that are running on top of them.

We also have to consider the touch points that exist outside of Kubernetes and how those will be automated in a way that lets a developer continue to move quickly. Take any common service launch as an example. Chances are the service is going to have some state that it needs to store, typically in a relational database. State isn’t something you necessarily want to manage inside Kubernetes (although admittedly, it’s getting better), so you might look to leverage something like Amazon RDS. Well now we’re in an ecosystem outside of Kubernetes. Are teams capable of also launching AWS resources to meet their needs or do they need to talk to an operations team? Does the operations team or someone else need to approve the spend for the AWS resources?

Tools like [Crossplane] (https://www.crossplane.io/) in the Kubernetes ecosystem allow us to give engineers the *capabilities* to perform those actions but is your organization giving them the *empowerment* to make those choices? And this is something simple like a relational database! It gets way more complex if we’re talking about things like API Gateways, ElastiCache or even Cognito.

Your thinking now might be “but I can’t let spend run out of control” and that’s a fair reaction. But the reason you might be making that argument is because **you** are responsible for the spend and not the creator of the spend. If the creator of the spend became responsible for it, that becomes a self-managing process as the spend starts to violate company norms. Again, this is where there has to be organizational buy-in to get the true effects of Kubernetes. Without that transfer of responsibility, the responsible party for the spend will continue to push back and create unnecessary hurdles and gates in order to relieve pressure on the spend side. But that relief comes at the cost of developer productivity. These types of battles are probably going on all throughout your organization and without a careful and specific process to address these things, they will forever limit your ability to truly get the most out of Kubernetes.

“But what about that time we had a massive outage because we let user X do action Y?” That’s a common refrain I hear when talking to people about doing something potentially new and scary. The truth however is yes, those things can happen. But they probably still happen even with draconic organization practices and policies. Jason Fried [once wrote]   (https://m.signalvnoise.com/dont-scar-on-the-first-cut/) that “policies are organizational scar tissue” and he’s never been more accurate.

Change is uncomfortable for everyone, but sometimes change is necessary to reap all the benefits and rewards out of a choice. If you’re choosing to implement Kubernetes, I urge you to also examine how your organization is going to change as a result. Do you have buy-in for the vision you have planned? How high does that buy-in go? This isn’t to say that your Kubernetes deployment can’t be successful, because it can be. But you might notice that you’ve got all of the same problems but just with a ton more complexity. And I’m sure that’s not what you were thinking of when you started your Kubernetes journey.